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Abstract 


This study adds to the currently limited evidence base on the efficacy of interventions targeting non- 
college-ready high school students by examining the impact of Kentucky’s Targeted Interventions (TI) 
program. We focus on interventions that students received under TI in the senior year of high school 
based on their 11th grade ACT test scores. Using difference-in-regression discontinuity and difference-in- 
difference designs with seven cohorts of 11th grade students, we find that, for an average per-student cost 
of about $600, TI significantly reduces the likelihood that students enroll in remedial course in both 2- 
and 4-year postsecondary institutions by 5—10 percentage points in math and 3-4 percentage points in 
English. These effects are similar among students who are eligible for free-or reduced-price lunch, Black 
and Hispanic students, students with remediation needs in multiple subjects, and students in lower- 
performing schools. Evidence also shows that TI increases the likelihood that students enroll in and pass 
college math before the end of the first year by four percentage points in 4-year universities. However, 


little evidence exists for TI affecting credit accumulation or persistence. 


ili 


1. Introduction 


Increasing the share of adults with postsecondary credentials is high on the national 
education agenda. This is reflected in national initiatives such as those put forth by the Obama 
administration,! as well as in education attainment goals set by many states.” For example, in 
Kentucky, which is the focus of this study, the Kentucky Council for Postsecondary Education 
(CPE) has set a goal for 60 percent of working-age Kentuckians to have high-quality 
postsecondary degrees or credentials by 2030 (CPE, 2019). 

Yet significant barriers to reaching these goals exist. Postsecondary graduation rates are 
low, and they have been stagnant for the last decade: Only about 30 percent of full-time 
community college students graduate within three years of initial enrollment and less than 60 
percent of 4-year university students graduate within six years.* Furthermore, low graduation 
rates disproportionately affect disadvantaged students. For example, among community college 
students, Black students are 30 percent less likely to graduate than White and Asian students. 

Numerous institutional and student factors may explain low postsecondary attainment 
(Holzer & Baum, 2017), but weak academic preparation before college matriculation is often 
cited as a key roadblock to student success (Bettinger, Boatman, & Long, 2013; Scott-Clayton, 
2011). Estimates, for instance, suggest that only a quarter of high school students who are 
assessed based on ACT college benchmarks appear fully college ready (ACT, 2014); not 
surprisingly then, the need for remediation in public postsecondary institutions is widespread, 
with 68 percent of 2-year college students and 40 percent of 4-year university students taking at 


least one remedial course (Chen, 2016; Bailey & Jaggars, 2016). 


' https://obamawhitehouse.archives. gov/blog/2009/07/14/investing-education-american-graduation-initiative 

? Foundations, such as the Lumina Foundation and the Bill & Melinda Gates Foundation, have also invested heavily 
in programs designed to promote the share of adults with postsecondary credentials in the labor force (Bailey, 2012). 
3 https://nces.ed.gov/programs/raceindicators/indicator RED.asp 


In this study, we evaluate the effects of a high school college readiness remediation 
program in Kentucky on students’ college outcomes using regression discontinuity (RD) and 
difference-in-differences (DD) methods. As the first state to adopt the Common Core State 
Standards (CCSS), Kentucky is also one of the earliest states to introduce statewide high school 
remediation for college. First implemented in the 2010—11 school year for high school math and 
reading and one year later for English, Kentucky’s Targeted Interventions (TI) program uses 
11th-grade ACT test scores to identify students for remediation. ACT tests have been mandatory 
for all 11th-grade students in Kentucky since 2006. Students scoring below 19 in math, 18 in 
English, or 20 in reading are considered not on track to be college-ready in those subjects when 
graduating high school. These students are required to receive intervention services or remedial 
support, with the goal of getting them ready to access “credit-bearing coursework without the 
need for developmental education or supplemental courses” in college (CPE, 2010, p.7), 
Kentucky’s working definition of college readiness. 

The rationale for addressing college-readiness gaps before college matriculation is 
straightforward: Traditional developmental education in college, which typically requires 
academically underprepared students to take up to three levels of remedial courses before they 
can start accumulating college credits, is both costly (Boatman & Long, 2018) and considered by 
many to be ineffective (e.g., Valentine, Konstantopoulos, & Goldrick-Rab, 2017). As of 2017, 17 
states offer statewide college-readiness remediation in high school (sometimes also called 
transition interventions) and 22 states offer local high school remediation programs (Barnett, 
Chavarin, & Griffin, 2018). The number of statewide programs has more than doubled since 
2013 (Barnett, Fay, Trimble, & Pheatt, 2013). Florida’s College and Career Readiness Initiative 


(FCCRI) (Mokher, Leeds, & Harris, 2018) and Tennessee’s Seamless Alignment and Integrated 


Learning Support (SAILS) program (Kane et al., 2019) are prominent examples of statewide 
high school remediation. 

Despite the widespread interest in high school remediation for college readiness, 
empirical evidence on its impact on college outcomes is relatively limited. Our study is one of 
four studies that look at one of the major efforts that states are adopting to help academically 
underprepared students succeed in higher education; moreover, the design and implementation of 
high school remediation programs vary considerably from state to state, and currently no 
evidence is available for the type of program implemented in Kentucky. As we describe in 
Section 3, Kentucky’s high school remediation program has a unique combination of 
programmatic and contextual features, and its experience will add significantly to the currently 
limited evidence base. 

Our study finds empirical evidence that Kentucky’s TI program significantly reduces the 
remedial course enrollment rate in both 2- and 4-year postsecondary institutions by 5-10 
percentage points in math and 3-4 percentage points in English. The TI program also boosts the 
enrollment rate in credit-bearing college math during the freshman year in 4-year universities, 
resulting in an overall increase of four percentage points in the likelihood of university students 
passing introductory college math by the end of the freshman year. The TI program reduces the 
likelihood of college remediation by similar magnitude among students who are eligible for free 
and reduced-priced lunch (FRL), Black and Hispanic students, students with remediation needs 
in multiple subjects, and students in lower-performing high schools. However, the TI program 
has no detectable effect on other measures of progress through college such as total credits 


earned by the end of the first two years in college or persistence. 


In what follows, we first describe the theory and empirical literature on high school 
remediation. This is followed by a description of Kentucky’s TI program. We next discuss the 
data, analytic samples, and methods. Impact estimates are presented in Section 5. We conclude 


this study with a discussion on how we may rethink high school remediation. 


2. Theory and Empirical Literature 


In recent years, efforts to improve the ways in which we help academically 
underprepared students acquire the requisite skills and knowledge for college-level instruction 
and contents have started to cut across postsecondary institutions and secondary schools. 
Traditionally, college remediation was predominantly delivered by postsecondary institutions 
through prerequisite developmental education models (Goudas & Boylan, 2012). Under such 
models, underprepared college students must take a series of remedial courses in math and 
English before proceeding to credit-bearing courses. 

By most accounts, prerequisite remediation appears not to be working as conceived. 
Despite total annual spending of $4 billion (Scott-Clayton & Rodriguez, 2015), traditional 
remediation sequences mostly have null—and in a few cases negative—effects on subsequent 
student outcomes such as passing gatekeeper courses, credit accumulation, and credential 
attainment (Bailey, Jaggars, & Scott-Clayton, 2013; Melguizo, Bos, Ngo, Mills, & Prather, 2015; 
Valentine, Konstantopoulos, & Goldrick-Rab, 2017).* Critics argue that traditional prerequisite 
remediation diverts resources and efforts away from making progress toward postsecondary 


credential completion, and that underprepared students are further disadvantaged by the 


4 For example, one study finds that, among students who were subject to lengthy remediation sequences, only 11 
percent and 29 percent of students referred to math and English remediation eventually passed introductory college 
courses in these subjects (Jaggars & Stacey, 2014). 


additional burden of remediation that has delivered no apparent academic benefits (Scott-Clayton 
& Rodriguez, 2015). 

The desire to minimize such “diversion effects” has led some reforms to push for 
remediation to be completed before college entry.> ° High school remediation programs are 
thought to benefit students by making clearer the skills students need to succeed in college.’ 
Among the 17 states that have statewide remediation programs, 13 use college placement 
assessments directly to screen high school seniors for intervention (Fay et al., 2017), thereby 
aligning expectations across the two education systems. High school remedial interventions are 
also frequently designed to target specific student deficiency areas identified by screener tests. 
Finally, by helping students become college-ready before college entry, high school remediation 
could potentially save students and the public costs in time and resources that college 
developmental education would require. 

But high school remediation also raises several concerns. The first concern is the 
opportunity cost represented by regular high school courses and other learning opportunities that 
are crowded out by mandatory remediation. In states and subjects where four years of courses are 
required for high school diploma, remediation in the senior year tends to displace more advanced 


regular courses such as pre-calculus, trigonometry and English IV (Kane et al., 2019; Mokher et 


5 Widespread reforms have been underway at the postsecondary level as well. These include changes to assessment 
and placement practices (such as the abolishment of placement assessments in Florida); accelerated remediation by 
combining two sequential remedial courses into one semester (e.g., Denver’s FastStart), reducing the number of 
remediation credit hours (e.g., Chabot’s accelerated English), or offering corequisites that allow students to enroll 
directly in college-level courses with paired support; and the development of distinct math pathways to better target 
what students need in their future jobs (Jaggars & Hodara, 2011). 

® Pre-college remediation also includes a few summer bridge programs. Existing empirical evidence is scant: 
whereas Bailey & Jaggars (2016) found a positive correlation between CUNY’s Start program and student 
outcomes, Chingos, Griffiths, & Mulhem (2017) found null effects of a low-cost online summer math program using 
a randomized experiment. 

’ Bettinger et al. (2013) point out that the misalignment between high school graduation requirements and the 
expectations of postsecondary institutions is a main reason why many students finish high school unprepared for the 
next step in their education. 


al., 2018).* Requirements for remediation in high school may also crowd out career technical 
education courses that may have higher relevance and labor market value to some students. 
Second and relatedly, the quality of remedial education that students receive at the expense of 
regular high school coursework is unclear. In both West Virginia and Florida (Mokher et al., 
2018; Pheatt, Trimble & Barnett, 2016), for instance, the emphasis of transition courses leaned 
toward test preparation for college placement assessments. In the case of Tennessee (Kane et al., 
2019), transition courses were modeled after college remedial courses. But college remedial 
courses have been criticized for their lack of relevant content (Bailey & Jaggars, 2016) and an 
instructional approach that emphasizes drilling and practice (Boatman, 2012; Grubb, 2013). In 
short, whether substituting regular high school coursework with remediation leads to a net gain 
in learning is unclear. 

Despite the fast growth of high school remediation programs, we are aware of only three 
evaluations of statewide programs. These programs differ in how students were selected for 
remediation, the objective of the intervention, and the intervention curriculum (see a summary of 
key differences and evaluation findings in Exhibit 1). Pheatt, Trimble, and Barnett (2016) 
examine the effect of West Virginia’s Transition Mathematics for Seniors in 2011-12 and 2012— 
13. Students were screened for remediation using a test administered in the junior year, and the 
primary goal of the math remedial course appeared to be preparing students to take and ideally to 
pass the COMPASS, the placement test that colleges used to refer students for developmental 
education. Using an RD design, the study finds no significant effect on passing the college 
remediation placement test and a negative effect on students’ likelihood of passing a college 


gatekeeper math course. 


8 Taking advanced courses in high school is shown to be associated with positive postsecondary outcomes (Attewell 
& Domina, 2008). 


Mokher et al. (2018) studies the impact of Florida’s high school remediation program, 
FCCRI, on student college outcomes. FCCRI identified students for remediation in two steps: 
Students who scored in the midrange of a 10th grade state standardized test (FCAT) were tested 
again in the 11th grade using college placement tests. The evaluated program was mandatory 
starting in 2011-12 but voluntary before then. Targeted students enrolled in transition courses in 
math and English that emphasized test preparation for college placement assessments such as the 
ACT or PERT. Using RD and a regression analysis, the study finds that FCCRI generally had no 
detectable effect on high school graduation, college enrollment, or the likelihood of enrolling in 
or passing college-level courses. 

Finally, Kane et al. (2019) evaluates Tennessee’s SAILS high school remediation 
program in math on student college outcomes. SAILS was launched in 201 1—12, and it gradually 
spread to more high schools but never reached more than 60 percent of all schools during the 
study period. In SAILS schools, ACT scores from the junior year were used to target students for 
both high school remediation and college placement. Successful completion of high school 
remedial math automatically exempted students from developmental education in college, and so 
it is not surprising that high school remediation reduced the enrollment of college remedial math 
by around 30 percentage points. Exemption from college remediation translated into small 
increases in college math enrollment, but it failed to increase the passing rate of college math or 
accelerate college credit accumulation. A post-intervention assessment also shows no significant 
improvement in intervention students’ math skills. 

It is worth noting that in all three states reviewed above, high school remediation 
programs have changed in important ways over time. In West Virginia, the math remediation 


curriculum studied by Pheatt et al (2016) was dropped after three years of full implementation. In 


Florida and Tennessee, the role of high school remediation diminished significantly after 
changes to developmental education at the college level. The abolishment of mandatory college 
placement testing in Florida in 2014 and the statewide introduction of corequisite developmental 
education in Tennessee in 2015 made high school remediation in those states less relevant, 
especially when the high school programs’ goals were limited primarily to test preparation or to 


fulfill college remediation requirements. 


3. Targeted Interventions in Kentucky 

In 2009, the Kentucky General Assembly passed Senate Bill (SB) | calling for a 
complete overhaul of Kentucky’s assessment and accountability system for PreK—12 education. 
Since then, the Commonwealth has embarked on a system of education reform—known as 


“Unbridled Learning”—designed to better monitor, document, and support progress toward the 


goal of college- and career- readiness. In response to SB1 and the strategic priorities 
subsequently established by the Kentucky Board of Education (KBE),’ the Kentucky Department 
of Education (KDE) and the Kentucky Council on Postsecondary Education (CPE) jointly 
developed the Targeted Interventions (TI) program (see Kentucky Statutes KRS158.6453 and 
KRS158.6459, and regulation 704 KAR3:305). 

The TI program is designed to monitor student progress on college- and career- readiness 
proficiencies, screen students who need help, and provide interventions that are targeted to fulfill 
those needs. This program has been in effect since the 2010—11 school year. Performance 


benchmarks are established for students in Grades 8, 10, and 11 using the EXPLORE, PLAN, 


° In 201 1, the KBE established four strategic priorities under Unbridled Learning: Next Generation Learners, Next 
Generation Professionals, Next Generation Support Systems, and Next Generation Schools and Districts. Together, 
these four priorities form the basis of Kentucky’s new accountability model, with all of its components fully 
implemented starting with the 2014-15 school year. 


and ACT tests, respectively. '° College readiness is benchmarked in three subject areas: math, 
English and reading. The program was gradually phased in over three years, starting with high 
school math and reading in 2010-11 and followed by high school English in 2011-12 and 
middle school math, reading, and English in 2012-13. 

Students scoring below the benchmarks are targeted to receive intervention services in 
their senior year. School districts determine the details of TI interventions, but each intervention 
cycle typically includes a diagnostic pretest, the development of instructional targets and their 
associated formative assessments, direct delivery of instruction, and a posttest. Supplemental 
instruction is often delivered through transition courses that are tailored toward individual 
students’ deficiency areas and aligned with the Kentucky Core Academic Standards (KY’s 
Common Core-aligned education standards). Transition courses can be either integrated into an 
existing course or a stand-alone course, and interventions can be delivered during regular school 
hours or as extended school services (ESS). Based on phone interviews with program 
administrators and surveys of teachers involved in supplemental instruction delivery, most 
transition courses make use of online curricula such as those developed by Dreambox, ALEX, 
and [XL in combination with teacher-developed materials, and test preparation does not appear 
to be the primary goal of these remedial interventions. More details about programmatic features 
and implementation details can be found in a companion cost study of the TI program conducted 


by Levin et al. (2020). 


'0 All three assessments were developed by ACT, Inc. as an integrated series of assessment called the Educational 
Planning and Assessment System (EPAS). EXPLORE was designed for 8th and 9th grade and PLAN was designed 
for the 10th grade. Each assessment provides predicted score ranges for subsequent assessments. EXPLORE and 
PLAN have been replaced by the ACT ASPIRE system since 2015. See 
https://files.eric.ed.gov/fulltext/ED510457.pdf for more details. 


Although the design of the TI program expressly states the intent to intervene early, our 
current study focuses on | 1th-grade students. This is because intervention reporting is mandatory 
for targeted 11th-grade students only. Using reported intervention data, we were able to detect— 
as one would expect if the program had been implemented as designed—a sharp jump in TI 
participation rates among students who score just below the benchmarks relative to students who 
score just above (see more details in Section 4). By comparison, intervention reporting is not 
mandatory for students in the 8th and 10th grades, and we failed to detect any sharp jumps in 
intervention participation rates around corresponding cutoff scores based on voluntarily reported 
intervention data.!! It is unclear whether the lack of discontinuity in intervention participation 
rates around cutoff scores is due to reporting issues, program implementation noncompliance, or 
both, rendering it impossible to interpret any “program impact” we may or may not find among 
8"_ and 10th-grade students. In addition, our study also focuses on math and English 
interventions only because student performance in these two subjects determines whether or not 
a student would be placed directly into college math and English courses that are required for 
graduation. 

The first row of Table | presents the number of 11th-grade students who scored at or 
above the ACT benchmarks (19 for math and 18 for English) and those who scored below the 
benchmarks in spring 2014 and 2015.'* More than 87,000 high school juniors took the ACT for 
the first time. Among those, more than 60 percent failed to meet the math benchmark and 40 
percent missed the English benchmark. 

About 56—57 percent of students who scored below the cutoff in either subject are 


documented to have participated in TI. On the other hand, between one-fifth to one-quarter of 


'! Data available upon request. 
'2 These are the first two cohorts of 11th-grade students for whom TI intervention data are available. 


students who scored at or above the ACT benchmarks also participated in TI (Table 1, Row 2). 
Unfortunately, we do not know what may have led to the observed program participation 
noncompliance (both “no shows” and crossovers). Communications with TI administrators 
suggest that other factors such as teacher input could have been considered in conjunction with 
ACT scores to refer students for TI. Students who failed the benchmarks on the first try might 
also retake the test to meet the benchmarks. Staff and parental resistance to high school 
remediation was also reported in our interviews with school administrators. Other possible 
factors like intervention exemptions and under-reporting cannot be ruled out. 

The rest of Table 1 and Table 2 present summary statistics on some of the characteristics 
of TI in order to present a more concrete picture of what high school remediation looked like in 
Kentucky. The bottom half of Table 1 describes the content and type of remediation that TI 
participants received by student group. Students could receive remediation in multiple content 
areas and forms. Among all 1 1th-grade students who participated in TI (column 1), 79 percent 
received remediation in math and 48 percent received remediation in English. Most students 
received high school remediation in the form of transition courses (63 percent), and close to one- 
third of students receive services through extended school services (ESS). It is interesting to note 
that half of students who failed to meet the math benchmark (column 3) also participated in 
English remediation, and that 78 percent of students who failed ACT English (column 5) 
received remediation in math. 

Some of the characteristics of TI remediation can be quantified using individual student- 
level data (Table 2). For example, an average intervention session lasted 55 minutes and was 
delivered four times a week. The most frequently-reported areas of deficiency included algebraic 


thinking, math reasoning, math computation, writing mechanics, and writing content. Around 


half (55 percent for math and 48 percent for English) of TI participants in our sample were 
judged by schools to have exited remediation successfully. 

The context, design, and delivery of high school remediation vary considerably from state 
to state (Barnett et al., 2018), and Kentucky’s TI has several unique characteristics. For example, 
most (12 out of 17) states that mandate statewide high school remediation automatically exempt 
students from developmental education in college after successful completion of high school 
remediation (Fay, Barnett, & Chavarin, 2017). Kentucky, by comparison, is one of five states 
that do not directly link high school remediation completion with developmental education 
exemption in college. Automatic exemption may be more efficient, but it mechanically produces 
an “effect” of reduction in college remedial course-taking without necessarily improving 
students’ academic proficiency, as Kane and colleagues (2019) found in Tennessee. 

In addition, high school remediation is typically designed to provide differentiated, not 
additional, learning opportunities for underprepared students. Because most states (e.g., Florida 
and Tennessee) require four years of math and English for high school graduation, remedial 
activities crowd out regular math and English courses that high school seniors would take. By 
comparison, Kentucky required only three years of math for high school graduation during the 
study period, and so Kentucky’s math remediation displaced less advanced, elective math 
courses (such as algebra II and college algebra). More importantly, one-third of targeted 1 1th- 
grade students in Kentucky received intervention through extended school services that averaged 
over 100 minutes per week, representing net gains in learning time outside of regular school 
hours. 

Finally, even though the ACT is used by both K-12 and postsecondary systems to assess 


college readiness and course placement in Kentucky, benchmarks do not align perfectly. 


Students considered college-ready by high schools may still be required to undertake 
developmental education in college. For example, although Kentucky high schools consider 
students who score 19 or higher on ACT math college-ready, only students scoring above 21 can 
take any college-level math courses. This contrasts with some states where different placement 
assessments are used for remediation referral in high school and college, or where assessment 


and placement policies perfectly align. 


4. Data and Methods 


4.1. Data and Sample Statistics 

We utilize longitudinal student-level administrative records provided by the Kentucky 
Center for Statistics (K YStats) that track student progress from high school through college 
between 2009 and 2018. These records include high school test scores, student background 
characteristics, and transcript data from both high school and postsecondary institutions in 
Kentucky, including both public and private postsecondary institutions. The data also include 
student-level records that document the interventions that remedial students received. The 
intervention data were first collected for the 2014 cohort of 11th-grade students, and they include 
information on intervention type, content area, curriculum and intensity. 

The individual student outcomes we focus on include whether students take remedial 
courses in college, whether they pass math and English credit-bearing courses in college, and 
persistence and credit accumulation by the second year of college. These outcomes are examined 


for 2- and 4-year institutions separately. !3 


'3 One limitation of our data is that we do not have information on out-of-state postsecondary enrollment for all 
cohorts. To assess the potential impact of missing data on our study, we examine college enrollment behavior among 
the 2013 cohort of 11th-grade students for whom we do have complete information on college enrollment through 
the National Student Clearinghouse. We find that out-of-state postsecondary enrollment accounts for less than three 


13 


The analytic sample of this study consists of seven cohorts of 11th grade students who 
took the ACT for the first time in the Spring of their junior year of high school between 2009 and 
2015. Each cohort consists of around 43,000 students. As depicted in Table 3, the 2010-2015 
cohorts of 11th grade students are subject to math TI and the 2011-2015 cohorts are subject to 
English TI (“Treated” cohorts that do not have individual student level intervention data are 
represented by solid dots, and cohorts that have individual level intervention data are represented 
by squares). The 2009 cohort represents the pre-treatment group for math and the 2009 and 2010 
cohorts represent the pre-treatment group for English (“Untreated” cohorts are represented by 
open dots in Table 3). 

Descriptive statistics are presented in Table 4, with each column representing a specific 
sample. Among all students, 85 percent are White and 48 percent are eligible for free/reduced 
price lunch (FRL) (column 1). The average ACT scores are very close to the college-ready 
benchmarks (Math=18.8, Reading=19.3 and English=18.4), with a standard deviation of about 5- 
6 points. The analytic samples for the RD analysis (that is, within 3 to 5 points around the ACT 
cut scores) have similar characteristics, with ACT scores slightly lower than the population 
average by around one point. 

4.2. Methods 

The TI program relies on ACT test scores to refer students for supplemental and remedial 
support. In a world with perfect compliance, all students with scores S below the cutoff c would 
be assigned to treatment (7=1), while those scoring at or above would not (7=0 if S=c). The 
treatment status would be a discontinuous function of ACT score: T=1(S<c). Under the 


assumptions that population average potential outcomes Y are continuous functions of the 


percent of students in the cohort around the ACT cutoffs, and there is no apparent discontinuity at the cut point in 
the likelihood of leaving the state. 


running variable (i.e., the ACT test score) at the cutoff, and that no determinants of the potential 
outcomes change abruptly at the cutoff other than treatment assignment, the causal effect of TI 
on students within the immediate vicinity of cutoff c could be reliably estimated using RD. This 
can be represented by regression equation (1), where x(.) is a function of the ACT score of 
student i, S;, that is centered around the cutoff such that negative values indicate scores below 
cutoff. 

Y; = QM + @4T; + K(S;) + @2T; * k(S;) + &. (1) 

Not all students scoring below the cutoff participate in TI (no-shows) and some students 
scoring above the cutoff receive treatment (cross-overs). Figure 1 depicts these two types of 
noncompliance using math intervention data from the 2014 and 2015 cohorts. Although the ACT 
score does not predict TI participation deterministically, it still holds that the probability of 
assignment to treatment changes discontinuously (by about 40 percentage points) at the cutoff. 
When the intended and actual assignment to TI deviate from each other, a, estimates the intent- 
to-treat (ITT) effect of high school remediation on student outcomes Y; instead of the treatment 
effect of the treated (TOT). 

Because ITT reflects the overall effect that a state could realistically expect from similar 
programs, and because we have intervention data for only two cohorts of 11th grade students, 
our study focuses on the ITT effect instead of the TOT effect that would be estimated using a 
fuzzy RD.'* The decision to focus on ITT is also related to two additional challenges in the 
current setting that have led us to use difference-in-RD as the preferred method to estimate the TI 


impact. 


'4 The TOT effect of TI on student outcomes is equivalent to the ITT effect scaled by the difference in remediation 
participation rates between students scoring just above and below the cutoff. 


The first challenge is related to the discreteness of the running variable and the resulting 
uncertainty associated with potential specification errors in the RD model. ACT scores are 
typically reported in whole number scores in a range of 1-36. Even with finer-grained underlying 
scores, the distribution of ACT scores is clearly lumped at several discrete score points (see Kane 
et al. [2019] which obtained finer-grained ACT scores for one cohort). When the running 
variable is discrete, it becomes infeasible to compare averages within arbitrarily small 
neighborhoods around the cutoff (Card & Lee, 2008). Impact estimates would instead have to 
rely on parametric assumptions about the relationship between the outcome and the running 
variable near the cutoff. Following the recommendation of Card and Lee (2008), researchers 
typically widen the confidence interval around the impact estimate to account for the additional 
uncertainty brought about by parametric assumptions. This is typically achieved by clustering the 
standard error at each distinct value of the running variable (CRV).!° With additional, pre- 
treatment data, however, we can also estimate a similar RD “effect” in the pre-treatment sample 
and take the difference between the pre- and post-RD estimates. To the extent that the underlying 
relationship between ACT scores and student outcomes in the absence of TI has not changed 
over time, difference-in-RD can produce specification-robust estimate of high school 
remediation on student outcomes. '© 

The second challenge has to do with the alignment between high school college-readiness 


benchmarks and course placement policy at the postsecondary level in Kentucky public 


'S The clustered variance estimator is known to be biased when the number of clusters is small (McCaffrey & Bell, 
2002), and more recent research provides convincing empirical and theoretical evidence that CRV standard errors 
generally understate the uncertainty related to specification errors and lead to substantially worse coverage 
properties than Eicker-Huber- White (EHW) heteroskedasticity-robust standard errors (Kolesar and Rothe, 2018). 
Because of the issues associated with CRV standard errors, we follow Kolesar and Rothe’s advice and use EHW 
standard errors when making inferences about estimated TI impacts on student outcomes. 

‘6 Difference-in-RD, sometimes also called difference-in-discontinuity, has been implemented to study a wide range 
of public policy topics. See, for example, Aucejo, Romano, & Taylor (2019); Dantas, Duarte, Neto, & Sampaio 
(2018); Jackson (2017); Grembi, Nannicini, & Troiano (2016); Xie (2015); Bettendorf, Folmer, & Jongen (2014). 
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institutions. Because students scoring above and below the math and English cut scores on the 
ACT also face different college remediation requirements, it is clear that TI participation is not 
the only “treatment” that differentiates students around the cutoffs. For example, in math, 
students scoring in the 16-18 range must (if they fail to retake and pass college placement 
assessments) take lower level remedial math courses in college (e.g., Basic Algebra and 
Mathematical Literacy) than students scoring in the 19-21 range, who are able to directly enroll 
in certain credit-bearing math courses with paired supplemental support (e.g., College Algebra 
Workshop). Any estimated RD effect on student college outcomes by Equation (1) would reflect 
the combined effect of both high school and college remediation. This is the type of scenario 
(i.e., the presence of confounding treatments around the cutoff) that typically calls for the use of 
difference-in-RD (Eggers, Freier, Grembi, & Nannicini, 2018). Because placement policies into 
college remedial sequences have remained constant for all 11th grade cohorts during the study 
period, we can use difference-in-RD to net out the effect that is due to differential college 
remediation requirements under the assumption that such effect is constant across all student 
cohorts. 

We conduct the usual validity checks on RD setup. The critical identifying assumption in 
an RD design is the continuity assumption: that absent the treatment, expected student outcomes 
would remain smooth through the cutoff. This assumption implies that determinants of potential 
outcomes other than treatment status are also smooth around the cutoff. Given these 
assumptions, any discontinuity in student outcomes can only be caused by the treatment. 

One way the continuity assumption may be violated is through the manipulation of the 
running variable such that students of different characteristics sort themselves into treatment and 


control groups. If TI were viewed as an extra burden, for example, more students might be 


expected to show up just at or above the cutoff than just below to avoid TI. This type of 
manipulation is unlikely because ACT tests are assessed without any teacher, student or principal 
influence and it is almost impossible for students to precisely target scores in the close vicinity of 
the cutoff.!’ The distribution of ACT test scores, centered around the cutoff, indeed shows no 
sign of score manipulation as it is smooth through the cutoff (Figure 2).!8 

Because students are unable to precisely achieve a score at or below the cutoff, we should 
not expect student attributes to change sharply around the cutoff. While this assumption cannot 
be definitively proven, as many individual and family characteristics are unobservable to the 
researchers, we can test for continuity of observable characteristics such as student demographic 
characteristics and FRL eligibility. As shown Table 5, within a bandwidth of three points around 
the cutoff, we do not generally detect significant discontinuity in student attributes around the 
cutoff scores with the exception of fewer females among students who scored just at or above the 
math benchmark than just below. However, controlling for gender does not meaningfully change 
our estimated TI effects. 

In addition to difference-in-RD, we also estimate a DD model in order to explore the 
potential effect of TI on students further away from the cutoff. The DD model: 

Yit = Ay + A, Tip + A2POStip + A2Tiz * Postit + Eit (2) 

compares the differences in student outcomes between students in different ACT score groups 
after the implementation of TI with similar differences before TI. a2 estimates the DD impact on 


student outcomes and T;, can accommodate various student groups depending on their score 


'7 Some students took the ACT more than once before the 12" grade. Unobserved factors, such as motivation, that 
influence test re-taking behavior may affect both the likelihood of passing the threshold after retesting and future 
education outcomes. To avoid this type of endogenous sorting, we use the required sitting for ACT in March of 
students’ junior years scores as the running variable when all students take the test at the same time. 

'8 We rely on graphical inspection because McCrary’s test for distributional discontinuity (2008) relies on local 
linear regressions that might lead to incorrect inferences when the running variable is discrete (Card & Lee, 2008). 
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range, with the reference group defined as students scoring just above the benchmarks to mimic 
the RD samples (i.e., ACT math 19-21 and ACT English 18—20).!? The validity of DD estimates 
of the TI effect relies on the assumption that, in the absence of TI, pre-treatment differences 
across student groups would have remained the same in the post-treatment period. In our 
analysis, we include two “treatment group” indicators in the vector Tir: students scoring just 
below the benchmarks (ACT math 16-18 or ACT English 15—17) and students scoring further 
below the benchmarks (ACT math 13—15 or ACT English 12-14). 
5. Findings 

Empirical findings are presented in four sets of tables. Tables 6a-6b display group means 
of student outcomes that we will refer to from time to time when discussing the estimated TI 
effects in order to put those estimates into context. Tables 7—9 present the estimated TI effects on 
high school completion and college enrollment rates. Although TI was not designed to affect 
high school-to-college transition outcomes, it is important to examine the extent to which the 
program may have altered the composition of entering college students, a potential mechanism 
through which TI may affect student college outcomes. Next, conditional on college enrollment 
Tables 8—9 present the estimated TI effects on various college outcomes. Finally, Table 10-12 
present findings for student subgroups as well as an exploratory analysis of the correlation 


between intervention type and student outcomes. 


'. The “just below” group resembles the RD sample that uses a bandwidth of three, and the a estimate for this 
group can be used as a further check on the robustness of difference-in-RD estimates. The main difference between 
the DD and difference-in-RD estimates is that our DD model assumes the student outcome to be constant within 
each student score group, whereas the RD framework allows for flexible relationship between the student outcome 
and the test score within each score group. The DD model is more restrictive, but it may also be less risky in 
situations where a limited number of data points are available to estimate a relationship. See a related discussion in 
Bloom (2003). 
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5.1. High School-to-College Transition 

The high school graduation rate among 11th grade students, at more than 90 percent, is 
high for all student groups in the analytic sample, which is broken down by ACT test score 
range, subject, and pre- and post-treatment periods in Tables 6a and 6b. The graduation rate in 
our samples is higher than the published high school graduation rate for Kentucky,” likely 
because Kentucky’s compulsory school leaving age at the time was 16 and most high school 
dropouts would likely have occurred before the spring of the 11th grade. Between one-third to 
two-thirds of 11th grade students in our sample enrolled in college immediately after high school 
graduation. Unsurprisingly, higher ACT scores are associated with higher high school graduation 
rate and college enrollment rate, and the association appears the strongest between ACT scores 
and the rate of enrollment in 4-year institutions. 

In order to understand whether the implementation of TI induced more marginal students 
to graduate high school and enroll in college, we present the estimated TI impact on these 
outcomes in Table 7. The estimated effects of math TI are shown in the top panel of the table, 
and the estimated effects of English TI are in the bottom panel. Difference-in-RD estimates, 
presented in the first three columns of Table 7 with varying bandwidths, show no detectable 
effect of high school math remediation on high school completion or college enrollment. High 
school English remediation, on the other hand, appears to lower the overall college enrollment 
immediately after high school graduation by 3-4 percentage points. However, the estimated 
impact becomes statistically insignificant when enrollment is examined for 2- and 4-year 


institutions separately. These results are largely consistent with DD estimates (the last two 


20 During the study period, Kentucky graduated 88 percent of all its high school students, with nearly 70 percent of 
the state’s school districts reporting graduation rates of 90 percent or higher. See 
http://www.americaspromise.org/resource/all-kids-how-kentucky-closing-high-school-graduation-gap-low-income- 
students 
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columns of Table 7) for students scoring just below the benchmark scores (16—18 in math and 
15-17 in English). Among students with much lower scores (13—15 in math and 12—14 in 
English), participation in high school remediation in either subject may have lowered their 2-year 
college enrollment by 2—3 percentage points, whereas participation in high school math 
remediation seems to have improved the chances of enrolling in 4-year universities. Overall, TI 
does not appear to have changed the composition of entering college students in significant 
ways. 
5.2. College Outcomes 

Figure 3 depicts the time trends of the rate of remedial course enrollment in college, the 
enrollment rate in introductory credit-bearing courses and the passing rate in credit-bearing 
courses. The trends are separately presented by ACT score range, subject and institution type (2- 
and 4-year). Among college students who scored below the 1 1th-grade ACT benchmarks, 
including those scoring far below the benchmarks, there is a clear decrease in the likelihood of 
remedial course enrollment in the first year in college. For example, among 2-year college 
students, even though students scoring just above the ACT benchmarks have been taking college 
remedial courses at a relatively steady rate over a seven-year period (about 10—14 percent), the 
percentage of college students scoring just below the ACT benchmarks (16—18 in math and 15— 
17 in English) who enroll in remedial courses has decreased from 50 percent to 30 percent in 
math and from 40 percent to 20 percent in English.”! We observe similar time trends in 4-year 


universities, where the remedial course enrollment rates of students scoring above and below the 


21 As the time between the original policy change and the cohort in question increases, it becomes more difficult to 
attribute change in developmental course-taking patterns in college to the introduction of TI as they could reflect 
other simultaneous efforts to reduce remediation at the postsecondary level. Alternatively, it is possible that the 
effects of TI increased over time as the program matured and schools had more experience with implementation. 
Because the biggest declines in developmental course-taking are in the later years, omitting them from our results 
yields attenuated estimates. Nonetheless, our estimates are still statistically significant when restricting the sample to 
smaller windows after the policy change. 
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ACT benchmarks have converged as a result of decreasing likelihood of students scoring below 
the cutoff enrolling in college remedial courses. 

Low-scoring students appear to use some of the extra time gained through reduced 
remediation needs to take credit-bearing courses in the freshman year, leading to an increase in 
college course enrollment rate in math and, to a lesser extent, in English. However, the passing 
rate of college math has not changed much for students scoring below the math benchmark 
relative to those scoring above the benchmark, suggesting that not all students who were induced 
to take college math were ready for those courses. 

We present the causal estimates of the TI effect on student college outcomes in Table 8 
for math and in Table 9 for English. The columns are organized by institution type (2-year vs. 4- 
year) and model specification (difference-in-RD with three bandwidths, followed by DD 
estimates for students just below and far below the ACT benchmarks). The rows are organized 
by college outcomes grouped into course-taking outcomes in college (the top two panels) and 
more general outcomes that describe student progress through college (the bottom panel). 

Difference-in-RD estimates in Table 8 show that, among students scoring in close 
vicinity around the ACT math benchmark, being referred to math remediation in high school 
reduces the likelihood of college math remedial course enrollment by 5—10 percentage points in 
2-year institutions and by seven percentage points in 4-year institutions. High school English 
remediation has similar, albeit weaker, effects, reducing the likelihood of college remedial 
English enrollment by 34 percentage points (Table 9). The difference-in-RD estimates can be 
better understood by an inspection of Figure 4, which plots student outcomes on the y-axis 
against ACT scores on the x-axis for students around the cutoff and compares their relationship 


in the pre-treatment (represented by circles) period with the relationship in the post-treatment 
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(represented by triangles) period. It reveals that TI helps reduce college remediation needs by 
significantly narrowing the remediation gap that existed before the introduction of TI between 
students just above and below the ACT cutoffs. 

The next six rows of Tables 8 and 9 present the estimated TI impact on the enrollment 
and passing rate of college courses. Difference-in-RD estimates show that, among 4-year 
university students scoring just below the ACT benchmark, TI increases the likelihood of 
enrollment in college math courses in the freshman year by four percentage points (Table 8). For 
these same students, TI also increases the overall passing rate of college math by a similar, four 
percentage points by the end of both the first and the second year in college. By comparison, 
there is no consistent evidence that TI has any impact on the enrollment and passing rates of 
college math among 2-year college students. Instead, math TI increases the first-year enrollment 
rate in college English by 3—7 percentage points, suggesting that 2-year college students may 
have opted to spend the extra time saved from math remedial courses to fulfill college English 
requirements. In contrast to math TI, there is no evidence that English TI has any detectable 
effects on the likelihood of enrolling in or passing college courses in either English or math in 
any types of institutions (Table 9). 

The bottom panel in Tables 8 and 9 show the estimated TI effects on credit accumulation 
by the end of the first two years in college and college persistence. There is no detectable TI 
effect on any of these measures of progress through college. The lack of statistical significance is 
not entirely surprising because the expected impact of TI on credit accumulation is much smaller 
than what statistical power would allow us to detect. Take math TI among 4-year university 
students as an example. Difference-in-RD estimates suggest a reduction of math remedial course 


enrollment rate of seven percentage points (Table 8) among students scoring just below the ACT 
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threshold (i.e., 16-18 if we assume a bandwidth of three), representing a 18 percent drop in 
remedial math enrollment in this student group.” These students, who would have had to take 
remedial math and earned an average of 23 credits by the end of the freshman year before the 
introduction of TI (Table 6a), can now earn three additional credits. Even if all of them took and 
successfully completed three additional college credits, the average number of earned credits for 
this student group (i.e., ACT math 16—18) would now be 0.18%*(23+3) + 0.82*23=23.54, a mere 
half-credit increase (equivalent to 0.06 standard deviation in earned credits by the end of 
freshman year, which is 9.09). 

The jump in the TI participation rate around the ACT benchmarks is 40 percentage 
points, and so the treatment effects among the treated (TOT) are likely larger than the intent-to- 
treat (ITT) effects reported above. A back-of-the-envelope calculation (1.e., dividing ITT 
estimates by 0.40) shows that, among students who participated in interventions, TI reduced the 
likelihood of remedial course enrollment by 13—25 percentage points in math and 8-13 
percentage points in English; it also increased the likelihood that treated students enrolled in and 
passed college math in 4-year universities by 10 percentage points. 

Tables 8 and 9 also present DD estimates of the TI impact for the same set of student 
outcomes as those investigated using difference-in-RD. As discussed in the methods section, 
here we compare the pre-post changes in student outcomes across student groups, with students 
scoring just above the ACT benchmarks (ACT math 19-21 or English 18—20) serving as the 
reference group. Each model includes indicators for two separate treatment groups: students 
scoring just below the benchmarks (ACT math 16—18 or English 15—17) and students scoring far 


below the benchmarks (ACT math 13-15 or English 12-14). The estimated TI impacts for both 


>? Forty percent of students in the ACT math 16-18 group took remedial math in the pretreatment period (Table 6a), 
and so a seven percentage points reduction is equivalent to a 100*7/40=18 percent change. 
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groups of low-scoring students are presented next to each other in Tables 8 and 9. The main 
purpose of the DD estimates for the “just below” students is to check the robustness of the 
difference-in-RD estimates of the TI effect, and these two sets of estimates are mostly consistent: 
DD estimates confirm that, among students who score just below benchmarks, being identified 
for high school remediation leads to significant decreases in the likelihood of college 
remediation in both English and math. It also increases the likelihood of college math course 
enrollment in 4-year universities. However, unlike difference-in-RD estimates, DD estimates fail 
to detect significant improvement in the passing rate of college math among 4-year university 
students. 

Results from DD also suggest that the estimated TI benefits are not limited to marginal 
students who just miss the ACT benchmarks. We find similar effects among lower-scoring 
students as well. For students scoring well below the benchmarks, TI reduces the likelihood of 
remediation in college by 10 or more percentage points in both math and English, and it 
increases the enrollment rate of credit-bearing math and English by 6—7 percentage points in both 
2- and 4-year institutions.”° 
5.3. Student and School Characteristics as Moderators 

In this section, we investigate possible variation in TI impact on student college outcomes 
among students who are underprepared in multiple subjects, minorities students, FRL-eligible 
students, and students in low-performing schools. There are reasons to believe that high school 


remediation may have differential impacts among these student subgroups. For example, the 


3 Tf high school remediation was successful and it reduced the need for remediation in college, as is the case with 
TI, it would also change the mix of students enrolled in college remedial courses simply because those who still 
need remediation despite having received interventions in high school are likely the lowest-performing. 
Consequently, a possible unintended consequence of a successful high school remediation program is a lower 
passing rate of college remedial courses. A quick calculation based on group statistics in Tables 6a and 6b supports 
this possibility: Before the implementation of TI, 65 percent of remedial math students in two-year colleges scoring 
just below the benchmark passed the course compared to 57 percent after the introduction of TI. 
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summary statistics in Table 1 show that many students who fail the ACT benchmark in one 
subject also tend to receive interventions in another subject. The demand for time and resources 
is likely higher for students who need remediation in multiple subjects than students who miss 
the benchmark in only one subject, potentially lowering the effectiveness of intervention in any 
single subject. On the other hand, since academic proficiency (and deficiency) is often correlated 
across subjects and cross-subject benefits of interventions are widely documented (e.g., Master, 
Loeb & Wyckoff, 2017), interventions in multiple subjects may reinforce one another and 
strengthen the effects of each intervention. 

TI may also have differential impact among minority and FRL-eligible students. This is 
because part of the effect of high school remediation is thought to be achieved through labeling 
students as not on track to be college-ready (Mokher et al., 2018). However, being labelled as not 
ready for college will motivate higher levels of student effort only when success is considered a 
goal that is attainable and when resources to help those students to catch up are available (Ou, 
2012; Papay et al., 2011). These conditions are often lacking for FRL and minority students. In 
addition, research suggests that Black students may be particularly susceptible to discouragement 
effect from being labelled as not college-ready (Dougherty, 2015). 

Finally, the effectiveness of TI could potentially vary by school performance level. The 
percentage of students failing to meet the ACT benchmark at the school level ranges from 0 to 
100. It is plausible that schools with large numbers of students failing to meet benchmarks could 
become overwhelmed by the need to tailor interventions to each student, so the intervention 
would become less targeted. On the other hand, higher demand for remediation may create 
economies of scale that allow schools to secure resources that are unaffordable to schools with 


few remediation students. 
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In Tables 10 and 11, we present results that investigate the potential heterogeneity of TI 
impact on student outcomes by the four student and school characteristics discussed above. To 
limit the size of the tables, we choose to focus on three first-year college outcomes that are most 
directly targeted by the TI program: the likelihood of students enrolling in college remedial 
courses and the likelihood of enrolling in and passing credit-bearing college math and English. 
These tables are structured in a way similar to Tables 8 and 9, with columns representing model 
specifications. We test for statistical equivalence between subgroup estimates and their 
corresponding population estimates, and bold and italicize cells where estimates are significantly 
different. 

Most subgroup estimates of the TI effects are qualitatively similar to the average effects 
reported in Tables 8 and 9. The most consistent finding is that TI significantly reduces the need 
for college remediation in nearly all student subgroups (with the exception of remedial English 
enrollment in 2-year colleges among FRL-eligible students and students who have failed 
benchmarks in multiple subjects). In very few instances, subgroup estimates are statistically 
different from population estimates. Specifically, math TI raises the enrollment rate of credit- 
bearing math during the freshman year among FRL-eligible students in 4-year universities by 
about 10 percentage points (compared to four percentage points on average), and English TI 
increases the enrollment rate of credit-bearing English among Black and Hispanic students in 2- 
year colleges by at least 12 percentage points (compared to an estimated zero effect on average). 

In some other instances, apparent differences in point estimates between student 
subgroups and the whole sample are not statistically significant due to large standard errors. 
Such differences concentrate mainly among the estimated TI impact on the likelihood of passing 


college-level courses. Notably, for students who fail benchmarks in multiple subjects and 
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students from high schools with the highest concentration (top quarter) of remedial students, 
being identified for math TI has a null effect on the likelihood of passing college math in 4-year 
universities (where the average effect is a four-percentage-point increase) and a negative effect 
in 2-year colleges (where the average effect is null). Among FRL-eligible students, TI has a 
stronger than average positive effect (nine percentage points) on the passing rate of college math 
in 4-year universities, but a statistically significant negative effect (-7 to -8 percentage points) in 
2-year colleges. For Black and Hispanic students, math TI fails to produce any detectable effects 
on college outcomes in 4-year universities where the population average effects are all 
significant. In contrast, in 2-year colleges where TI has null effects in the full sample, TI 
increases in the likelihood of college course enrollment of Black and Hispanic students by more 
than 10 percentage points in both math and English. Although the increase in the college English 
enrollment rate leads to a 6-13 percentage points increase in the passing rate of college English, 
a similar increase in college math enrollment fails to translate into discernible gains in the 
passing rate of college math. 
5.4. Intervention Type 

TI interventions were reported under three categories: transition courses, extended school 
services (ESS) and “other” interventions. Of particular interest to us is the impact of TI delivered 
through ESS and how it compares to the effect of transition courses. This is because ESS 
interventions, which average more than 100 minutes per week and affecting one-third of TI 
students, provides additional learning time for students who need help. This contrasts with other 
high school remediation programs (such as those evaluated in West Virginia, Florida, and 
Tennessee), which typically replaces regular high school coursework with remediation without 


adding additional learning opportunities. It is hardly conceivable that, without spending more 
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time to study, underprepared students would be able to catch up with their more advanced peers. 
Indeed, abundant evidence shows positive causal effects of additional instruction and learning on 
student outcomes.”4 

Unfortunately, the investigation of potential heterogeneous effect by intervention type is 
hampered by two factors. First, our reliance on difference-in-RD to identify TI effects prevents 
us from estimating separate TI effects for each intervention type. Moreover, a fundamental 
concern is that the choice of intervention type is likely endogenous: Extra learning opportunities 
may have been provided to students after regular hours for those who needed more help based on 
factors that researchers cannot observe. 

With these concerns in mind, we conducted an exploratory analysis by estimating an 
ordinary least squares regression for each college outcome, TI subject and postsecondary 
institution type, with standard errors clustered at the school level. Student outcome is regressed 
on indicator variables for ESS and other interventions (with transition courses as the reference 
intervention type), ACT test scores, FRL eligibility, gender and race/ethnicity. The analytic 
samples include the 2014 and 2015 cohorts of 11th-grade students who participated in the TI 
program and have available intervention data. Estimated coefficients on intervention type 
indicators are reported in Table 12. Results are mixed. Relative to transition courses, ESS is 
associated with larger increases (by four percentage points) in the passing rate of college math in 
4-year universities. For English, however, ESS students are more likely than students who take 


transition courses to enroll in college remedial English (three percentage points in 4-year 


4 For example, Jacob and Lefgren (2004) find that summer school classes targeting low-performing students in 
Chicago elementary schools increase academic achievement in reading and mathematics and that these positive 
effects remain substantial at least 2 years following the completion of the program. Taylor (2014) and Cortes, 
Goodman and Nomi (2015) both find that adding a remedial math course to a regular math course in middle school 
has positive effects on test scores, though the immediate effects decayed quickly in a couple of years. And Figlio, 
Holden and Ozek (2018) find significant benefits of additional reading instruction provided through extended school 
days on reading test scores. 
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institutions and five percentage points in 2-year colleges); ESS students are also four percentage 
points less likely to take introductory English in 2-year institutions, but the difference is not 
statistically significant. 


6. Discussion and Conclusion 


This study is part of a growing body of evidence that explores linkages from high school 
to college. Understanding these linkages is especially important given that high school and 
college are increasingly treated as a continuum as opposed to distinct educational processes in 
preparing students for college-level education. On the other hand, postsecondary institutions 
frequently admit students deemed not college ready, but react to the preparation level of students 
with various initiatives, including the developmental course requirements used as a dependent 
variable here.*° On the other hand, getting all students ready for college has become the mantra 
for K-12 schools so college outcomes are an important indicator for the success of the K-12 
system. 

This study adds to the limited evidence about the extent to which high school remediation 
can help academically underprepared students succeed in college. We find evidence consistent 
with the goal set forth by Kentucky for the TI program: Being assigned to TI is estimated to lead 
to reductions in the likelihood that students take remedial courses in college and increases in the 
likelihood that 4-year university students will take and pass college math. 

The TI program findings are sizable. The intent-to-treat estimates suggest that TI reduces 


the likelihood of remedial course enrollment by 5—10 percentage points in math and 3-4 


5 The co-requisite model is another strategy by which colleges react to students who are deemed not college ready. 
Under this strategy, students enroll directly in college-level courses with a paired support course. In 2016, over a- 
third of community colleges across the nation were offering co-requisite options in English and 16% in math (Scott- 
Clayton, 2017). The adoption of the co-requisite model has continued to grow at a quick pace, and in 2018, 15 states 
recommended or mandated co-requisite reforms (Rutschow, 2019). Kentucky introduced the co-requisite model in 
its community college system in 2019. 
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percentage points in English, and the program is estimated to increase the likelihood that 4-year 
university students take and pass college math during the freshman year by four percentage 
points. Importantly, only half of the students who scored below the ACT benchmarks were 
treated, suggesting treatment-on-the treated effects that are roughly twice as large. The estimated 
effects are similar in terms of reducing the need for college remediation among disadvantaged 
students, students in low-performing schools as well as students scoring far below college-ready 
benchmarks. 

These effect estimates are also sizeable relative to the cost of the program and remedial 
courses (as an assumed necessity if the program did not exist). The resources associated with 
developing and implementing TI amounted to about $600 per treated student (Levin et al., 2020). 
By comparison, each 3-credit remedial course costs between $830 (in community colleges in 
Tennessee, Belfield, Jenkins, & Lahr [2016]) and $1,500 (national average across all institution 
types, Barry & Dannenberg [2016]). 

On the whole, our findings suggest that the TI program helps address college readiness 
issues at the front end of the college pipeline. We did not find significant evidence that it had 
positive effects on progress through college (credit accumulation and college persistence). But, 
these findings should be interpreted with caution given that the estimated impact on early college 


outcomes may be too small to produce discernible changes in later college outcomes. 
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Figure 1. Math Targeted Intervention (TI) Participation by ACT Math Score 
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Figure 2. Distribution of ACT Test Scores in Kentucky, by Subject 


g Math g Reading 
‘= ‘* 
cs rs i il 
i] o- 
-20 -10 0 10 20 -20 -10 0 10 20 
Centered score Centered score 
3 English 
>8 
gS 
g 
cS 
o 
20 10 0 10 20 
Centered score 


40 


Take developmental math 


| a 


8 


6 


Take ee math 
: 4 


| 


_ 


T T T T T 
2009 2010 2011 2012 2013 2014 2015 
ACT test year 


Take developmental math 


————e 


T PTE TE T 1 
2009 2010 2011 2012 2013 2014 2015 
ACT test year 


8 


6 


2 


bg developmental math 
4 


Math: 2 year institutions 


Take credit-bearing math 


o—e—_»_e_*_e—_-e 
+—_*—__»_, + —* 


——— 


8 


6 


Take credit-bearing math 
2 4 


0 


Pass credit-bearing math 


8 


Pass credit-bearing math 
2 


ra a ae 


0 


T 7 t r T T 
2009 2010 2011 2012 2013 2014 2015 
ACT test year 


Math: 4 year institutions 


Take credit-bearing math 


lige 
= 


6 


Take credit-bearing math 
2 #4 


0 


T T T T T T 
2009 2010 2011 2012 2013 2014 2015 
ACT test year 


T T T T T ™ 
2009 2010 2011 2012 2013 2014 2015 
ACT test year 


Pass credit-bearing math 


8 


6 


o_, +. _*_*—_ 


—_, = 
1 


T rT T T T 7: 
2009 2010 2011 2012 2013 2014 2015 
\CT test year 


Pass credit-bearing math 
2 44 


tv) 


—e— ACTM 22-24 —®— 19-21 —#— 16-18 —4— 13-15 


Take developmental English 


een 


2009 2010 2011 2012 2013 2014 2015 
ACT test year 


ish 
1 


mental Engl 
2 o 6 ee 


Take develo; 


Take developmental English 


bpmental English 
4 6 8 1 


2 


T T T 7 7 T 
2009 2010 2011 2012 2013 2014 2015 
ACT test year 


Take develo 
0 


English: 2 year institutions 


Take credit-bearing English 
—3— 3 


— 


Take credit-bearing English 
24 8 8 1 


0 


2009 2010 2011 2012 2013 2014 2015 
ACT test year 


English: 4 year institutions 


Take credit-bearing English 


> 


Take credit-bearing English 
2a&8 8 4 


0 


T T T 7 T 1 
2009 2010 2011 2012 2013 2014 2015 
ACT test year 


Pass credit-bearing English 


1 


8 


=———<— —— = 
=—s—s—_.»_s = * 


————— et 


2 4 «6 


ae credit-bearing English 


2009 2010 2011 2012 2013 2014 2015 
ACT test year 


Pass credit-bearing English 


SSS 


T T T T T T 
2009 2010 2011 2012 2013 2014 2015 
ACT test year 


Pass credit-bearing English 
a £8 4 


0 


—e— ACTE 21-23 —@— 18-20 —e— 15-17 —*#— 12-14 


Figure 3. Trend in College Outcomes, by Institution Type, Subject, and ACT Scores: 2009-2015 
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Figure 4. College Outcomes by ACT Test Scores, by Subject and Institution Type 
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Exhibit 1. Existing Studies of Statewide High School Remediation: Program Features and 
Evaluation Findings 


State 


WV (Pheatt et al. 2016) 


FL (Mokher et al., 2018) 


TN (Kane et al., 2019) 


Subject 


Math 


Math 
and 
English 


Math 


Screener 


Not directly 
aligned with 
college 
placement 


2-stage 
screening, with 
2™4 stage using 
college 
placement tests 


Same as college 
placement tests 


Main objective 


Test preparation for 
college placement 
assessments 


Test preparation for 
college placement 
assessments 


Automatic exemption 
from college remediation 
upon successful 
completion of high school 
remediation 


Findings 


No effect on passing the placement 
test and negative effect on passing 
college math. 


No effect on the likelihood of 
college remediation placement, or 
the enrollment and passing rates of 
college-level courses. 


Lowered college remediation 
placement rate by 30 percentage 
points, small increase in enrollment 
rate in college math, but no impact 
on the passing rate of college math 
or credit accumulation. 
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Table 1. Targeted Intervention (TI) Participation Rate, by Test Subject, 
Intervention Content and Type: 2014 and 2015 11th Grade Students 


Total students with valid ACT score 
Percent with record of TI 
Among TI participants: 


CONTENT 
Math (%) 
English (%) 

TYPE 


Extended school services (ESS) (%) 


Course (%) 
Other (%) 


All 


87,811 
41 


79 
48 


31 
63 
20 


Math English 

At or At or 

above Below above Below 
cut cut cut cut 

score score score score 

35,040 52,771 49,373 38,438 
18 56 28 57 
47 85 80 78 
39 50 18 67 
54 26 37 27 
38 69 55 68 
14 22 17 23 


Note: Less than 1.5 percent of Tl participants have missing information on Tl type. Percentages add up to 


more than 100 percent because students can receive Tl in more than one content area and type. 
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Table 2. Characteristics of Targeted Interventions (TI), by Subject: 2014 and 


2015 11th Grade Students 


Deficiency area (% of intervention by subject) 
Algebraic Thinking 
Math Reasoning 
Math Computation 
Writing Mechanics 
Writing Content 


Other TI characteristics 
Duration (Minutes/Session) 
Frequency (Number of Sessions/Week) 
Teacher-developed materials (%) 
In-person delivery (%) 


Successful exit of TI (%) 


Math 


English 


— Not applicable 
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Table 3. Pre- and Post-Treatment 1 1th Grade Cohorts 


2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15 


2015-16 


Math ° ® e e e 


English ° ° e e e 


Note: Years refer to the time when a student was in the 11th grade. 
° Pre-treatment 


e Post-treatment without intervention data 


Post-treatment with intervention data 
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Table 4. Descriptive Statistics of Study Samples: 2009-2015 


ruil RD Sample Math RD Sample English 
sania BW=3 BW=4 BW=5 BW=3 BW=4 BW=5 
ACT-M 16-21. ACT-M15-22  ACT-M 14-23 ACT-E 15-20 ACT-E 14-21 ACT-E 13-22 
Demographics 
Female (%) 50 52 52 51 51 51 51 
White (%) 85 85 84 84 85 85 85 
Black (%) 10 9 11 11 10 10 10 
Hispanic (%) 5 5 5 5 5 5 5 
FRL (%) 48 49 52 52 50 50 49 
Test Scores 
ACT Math 18.8 17.6 17.3 17.3 17.7 17.8 17.9 
(4.5) (1.6) (2.1) (2.6) (2.8) (3.0) (3.1) 
ACT Reading 19.3 18.7 18.2 18.1 18.0 18.1 18.2 
(5.9) (4.6) (4.7) (4.9) (3.6) (3.8) (4.0) 
ACT English 18.4 17.9 17.2 17.0 17.4 17.5 17.6 
(6.3) (4.5) (4.9) (5.1) (1.8) (2.4) (2.8) 


Note: Standard deviation in parentheses. 
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Table 5. Covariate Balance Check 


Math English 

All 2 year 4 year All 2 year 4 year 

Female -0.04** -0.07** -0.05* 0.00 0.03 -0.02 
(0.02) (0.03) (0.03) (0.02) (0.03) (0.03) 

White 0.01 -0.02 0.03* 0.00 0.01 -0.01 
(0.01) (0.02) (0.02) (0.01) (0.02) (0.02) 

Black -0.01 0.01 -0.02 -0.00 -0.02 -0.01 
(0.01) (0.01) (0.02) (0.01) (0.02) (0.02) 

Hispanic 0.00 0.01 -0.01 -0.02** -0.02 0.00 
(0.01) (0.01) (0.01) (0.01) (0.01) (0.01) 

FRL -0.01 -0.01 0.00 -0.01 -0.05* -0.01 
(0.02) (0.03) (0.02) (0.02) (0.03) (0.03) 


Note: * and ** denote statistical significance at the 0.10 and 0.05 levels, respectively. Standard errors in 
parentheses. RD estimates assume linear association between the running variable and outcomes with a 


bandwidth of three. Results are robust to other bandwidth choices. 
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Table 6a. Group Means of Student Outcomes, by Treatment Period and ACT Math 
Score Range: 2009-2015 


Pre-Treatment Period Post-Treatment Period 
(2009) (2010-2015) 
ACT ACT ACT ACT ACT ACT 

Student Outcomes 19-21 16-18 13-15 19-21 16-18 13-15 
High school-to-college transition 
High school graduate (%) 98 96 92 98 96 92 
Enroll in college immediately after high school (%) 71 57 34 67 52 30 

2-year institution (%) 22 26 22 23 26 20 

4-year institution (%) 49 31 12 44 26 9 
Two-year college outcomes 
Take remedial math by end of 1st year (%) 12 52 60 12 42 54 
Pass remedial math by end of 1st year (%) 9 34 31 7 24 27 
Take credit-bearing math by end of 1st year (%) 71 40 15 71 45 21 
Pass credit-bearing math by end of 1st year (%) 42 24 7 46 25 9 
Take credit-bearing math by end of 2nd year (%) 78 51 24 78 54 30 
Pass credit-bearing math by end of 2nd year (%) 51 34 13 53 33 15 
Total credits earned by end of 1st year 22 17 12 24 19 13 
Total credits earned by end of 2nd year 31 24 15 31 24 16 
Return for 2nd year of college (%) 66 61 51 64 57 48 
Four-year university outcomes 
Take remedial math by end of 1st year (%) 11 40 54 9 29 41 
Pass remedial math by end of 1st year (%) 7 25 30 5 18 23 
Take credit-bearing math by end of 1st year (%) 67 46 28 68 51 33 
Pass credit-bearing math by end of 1st year (%) 42 25 11 43 27 11 
Take credit-bearing math by end of 2nd year (%) 76 60 43 77 64 47 
Pass credit-bearing math by end of 2nd year (%) 51 36 20 52 37 20 
Total credits earned by end of 1st year 27 23 18 28 24 19 
Total credits earned by end of 2nd year 43 36 28 43 36 27 


Return for 2nd year of college (%) 83 77 69 79 73 65 


Table 6b. Group Means of Student Outcomes, by Treatment Period and ACT English 
Score Range: 2009-2015 


Pre-Treatment Period 


Post-Treatment Period 


(2009-10) (2011-2015) 
ACT ACT ACT ACT ACT ACT 

Student Outcomes 18-20 15-17 12-14 18-20 15-17 12-14 
High school-to-college transition 
High school graduate (%) 97 95 94 97 96 94 
Enroll in college immediately after high school (%) 66 55 40 62 49 35 

2-year institution (%) 26 28 25 26 26 23 

4-year institution (%) 40 27 16 35 23 13 
Two-year college outcomes 
Take remedial English by end of 1st year (%) 5 30 43 2 18 29 
Pass remedial English by end of 1st year (%) 3 19 27 1 11 18 
Take credit-bearing English by end of 1st year (%) 82 63 44 85 68 54 
Pass credit-bearing English by end of 1st year (%) 59 43 28 62 47 34 
Take credit-bearing English by end of 2nd year (%) 86 71 54 88 74 62 
Pass credit-bearing English by end of 2nd year (%) 64 51 37 66 53 42 
Total credits earned by end of 1st year 21 17 13 22 18 15 
Total credits earned by end of 2nd year 28 22 17 28 22 18 
Return for 2nd year of college (%) 64 59 53 61 55 50 
Four-year university outcomes 
Take remedial English by end of 1st year (%) 1 23 44 0 13 29 
Pass remedial English by end of 1st year (%) 0 15 28 0 8 18 
Take credit-bearing English by end of 1st year (%) 86 80 72 87 84 79 
Pass credit-bearing English by end of 1st year (%) 58 51 41 60 52 45 
Take credit-bearing English by end of 2nd year (%) 90 87 82 91 89 85 
Pass credit-bearing English by end of 2nd year (%) 64 58 50 65 58 51 
Total credits earned by end of 1st year 25 22 19 27 23 20 
Total credits earned by end of 2nd year 39 34 29 40 34 28 
Return for 2nd year of college (%) 79 75 71 75 71 65 
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Table 7. Estimated Effect of Targeted Interventions (TI) on High School to College 
Transition Outcomes, By Intervention Subject and Model 


Difference-in-RD estimates 


BW=3 BW=4 BW=5 DD estimates 
Based on ACT math cutoff ACT 16-18 ACT 13-15 
High school graduate 0.01 0.01 0.01* 0.00 -0.00 
(0.01) (0.01) (0.01) (0.00) (0.00) 
Enroll in college immediately after high school 0.00 0.00 -0.00 -0.00 0.01 
(0.01) (0.01) (0.01) (0.01) (0.01) 
Enroll in 2-year institutions 0.00 -0.00 0.00 -0.01* -0.02*** 
(0.01) (0.01) (0.01) (0.01) (0.01) 
Enroll in 4-year institutions 0.00 0.00 -0.00 0.01 0.03*** 
(0.01) (0.01) (0.01) (0.01) (0.01) 
Based on ACT English cutoff ACT 15-17 ACT 12-14 
High school graduate 0.00 0.01 0.01 0.01 0.00 
(0.01) (0.01) (0.01) (0.00) (0.00) 
Enroll in college immediately after high school -0.04**  -0.04**  -0.03* -0.02*** -0.02*** 
(0.02) (0.02) (0.01) (0.01) (0.01) 
Enroll in 2-year institutions -0.02 -0.02 -0.01 -0.02*** -0.03*** 
(0.02) (0.02) (0.01) (0.01) (0.01) 
Enroll in 4-year institutions -0.02 -0.02 -0.01 -0.00 0.01 
(0.02) (0.02) (0.01) (0.01) (0.01) 


Note: *, ** and *** denote statistical significance at the 0.10, 0.05 and 0.01 levels, respectively. Standard errors in 
parentheses. Difference-in-RD estimates assume linear association between the running variable and outcomes. DD 


estimates use students scoring up to three points above the cutoff as the comparison and control for student ACT scores, 


FRL-eligibility, gender and race/ethnicity. 
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Table 8. Estimated Effect of Targeted Interventions (TI) in Math on College Outcomes 


2 year 4 year 
Difference-in-RD DD Difference-in-RD DD 
ACT 16- ACT 13- ACT 16- ACT 13- 
BW=3 BW=4 BW=5 18 15 BW=3 BW=4 BW=5 18 15 
College math outcomes 
Take remedial math by end of 1st year -0.05* -0.09*** -0.10*** -0.12*** -0.10*** -0.07*** -0.07*** -0.07*** -0.10***  -0.14*** 
(0.03) (0.02) (0.02) (0.01) (0.01) (0.02) (0.02) (0.02) (0.01) (0.02) 
Take credit-bearing math by end of ist year 0.04 0.03 0.02 0.04** 0.07*** 0.04 0.04* 0.04** 0.04***  0.06*** 
(0.03) (0.03) (0.02) (0.02) (0.02) (0.03) (0.02) (0.02) (0.01) (0.02) 
Take credit-bearing math by end of 2nd year 0.04 -0.00 -0.00 -0.00 0.05*** 0.03 0.01 0.02 0.03** 0.06*** 
(0.03) (0.03) (0.03) (0.02) (0.02) (0.02) (0.02) (0.02) (0.01) (0.02) 
Pass credit-bearing math by end of 1st year -0.00 -0.02 -0.03 -0.03 -0.01 0.04* 0.04* 0.04* 0.01 0.00 
(0.03) (0.03) (0.02) (0.02) (0.02) (0.02) (0.02) (0.02) (0.01) (0.01) 
Pass credit-bearing math by end of 2nd year 0.01 -0.03 -0.04* -0.04*** -0.00 -0.00 -0.02 -0.02 -0.02 -0.01 
(0.03) (0.02) (0.02) (0.01) (0.01) (0.02) (0.02) (0.02) (0.01) (0.01) 
Cross-subject outcomes 
Take credit-bearing English by end of 1st year 0.07***  0.05** 0.03* 0.03***  0.06*** -0.00 0.00 0.01 0.03***  0.05*** 
(0.02) (0.02) (0.02) (0.01) (0.01) (0.02) (0.02) (0.01) (0.01) (0.01) 
Pass credit-bearing English by end of 1st year 0.05 0.03 0.03 0.02 0.03* 0.01 0.01 0.01 0.01 0.03* 
(0.03) (0.03) (0.02) (0.02) (0.02) (0.02) (0.02) (0.02) (0.01) (0.02) 
General college outcomes 
Total credits earned by end of 1st year -0.03 -0.19 0.12 -0.38 -0.21 0.43 0.58 0.57 -0.04 -0.27 
(0.69) (0.60) (0.56) (0.37) (0.38) (0.44) (0.39) (0.37) (0.23) (0.30) 
Total credits earned by end of 2nd year 1.35 0.08 0.60 -0.69 0.12 0.94 1.16 1.12 0.46 -0.04 
(1.59) (1.40) (1.30) (0.88) (0.89) (1.17) (1.04) (0.97) (0.60) (0.80) 
Return for 2nd year of college 0.01 -0.01 -0.00 -0.02 -0.01 0.00 -0.00 -0.00 0.00 0.00 
(0.03) (0.03) (0.02) (0.02) (0.02) (0.02) (0.02) (0.02) (0.01) (0.02) 


Note: *, ** and *** denote statistical significance at the 0.10, 0.05 and 0.01 levels, respectively. Standard errors in parentheses. Difference-in-RD estimates assume 
linear association between the running variable and outcomes. DD estimates use students scoring up to three points above the cutoff as the comparison and control 
for student ACT scores, FRL-eligibility, gender and race/ethnicity. 
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Table 9. Estimated Effect of Targeted Interventions (TI) in English on College Outcomes 


2 year 4 year 
Difference-in-RD DD Difference-in-RD DD 
ACT 15- ACT 12- ACT 15- ACT 12- 
BW=3 BW=4 BW=5 17 14 BW=3 BW=4 BW=5 17 14 
College English outcomes 
Take remedial English by end of 1st year -0.02 -0.04** -0.04** -0.11*** = -0.14*** -0.00 -0.03* -0.03** -0.10*** = -0.15*** 
(0.02) (0.02) (0.02) (0.01) (0.01) (0.02) (0.02) (0.01) (0.01) (0.01) 
Take credit-bearing English by end of 1st year 0.03 -0.01 -0.01 0.02* 0.07*** 0.01 0.00 0.02 0.03*** 0.06*** 
(0.03) (0.02) (0.02) (0.01) (0.01) (0.02) (0.02) (0.02) (0.01) (0.01) 
Take credit-bearing English by end of 2nd year 0.03 0.00 -0.00 0.00 0.03** -0.03 -0.04* -0.01 -0.02* -0.02 
(0.03) (0.02) (0.02) (0.01) (0.01) (0.03) (0.02) (0.02) (0.01) (0.01) 
Pass credit-bearing English by end of 1st year 0.04 0.01 -0.00 0.01 0.04*** -0.01 -0.03 -0.00 -0.01 0.02 
(0.03) (0.02) (0.02) (0.01) (0.01) (0.03) (0.02) (0.02) (0.01) (0.02) 
Pass credit-bearing English by end of 2nd year 0.01 0.00 -0.01 -0.00 0.01 -0.02 -0.02 -0.01 -0.01 0.00 
(0.03) (0.02) (0.02) (0.01) (0.01) (0.03) (0.02) (0.02) (0.01) (0.01) 
Cross-subject outcomes 
Take credit-bearing math by end of 1st year 0.03 0.00 -0.01 0.02 0.05*** 0.02 0.02 0.03 0.01 -0.01 
(0.03) (0.02) (0.02) (0.01) (0.01) (0.03) (0.02) (0.02) (0.01) (0.02) 
Pass credit-bearing math by end of 1st year -0.03 -0.04 -0.04 -0.02 -0.02 0.01 0.01 0.02 -0.01 -0.03** 
(0.03) (0.02) (0.02) (0.01) (0.01) (0.03) (0.02) (0.02) (0.01) (0.01) 
General college outcomes 
Total credits earned by end of 1st year 0.62 0.05 -0.04 -0.10 0.28 -0.40 -0.42 0.07 -0.31 -0.20 
(0.63) (0.52) (0.46) (0.27) (0.29) (0.52) (0.43) (0.38) (0.22) (0.27) 
Total credits earned by end of 2nd year 0.85 0.08 -0.23 -0.28 0.28 -2.26 -2.29** -0.77 -0.98* -0.91 
(1.45) (1.21) (1.07) (0.64) (0.66) (1.39) (1.16) (1.03) (0.58) (0.71) 
Return for 2nd year of college 0.02 0.01 0.00 -0.01 -0.01 -0.03 -0.03 -0.01 -0.02* -0.03* 
(0.03) (0.02) (0.02) (0.01) (0.01) (0.03) (0.02) (0.02) (0.01) (0.01) 


Note: *, ** and *** denote statistical significance at the 0.10, 0.05 and 0.01 levels, respectively. Standard errors in parentheses. Difference-in-RD estimates assume 
linear association between the running variable and outcomes. DD estimates use students scoring up to three points above the cutoff as the comparison and control for 
student ACT scores, FRL-eligibility, gender and race/ethnicity. 


33 


Table 10. Estimated Effect of Targeted Interventions (TI) in Math on College Outcomes, by Student and School Characteristics 


2 year 4 year 
Difference-in-RD DD Difference-in-RD DD 
BW=3 BW=4 BW=5 ACT 16-18 ACT 13-15 BW=3 BW=4 BW=5 ACT 16-18 ACT 13-15 
Students who missed ACT cutoff in addition to math 
Take remedial math by end of 1st year -0.06 -0.08*** -0.09***  -0.09*** -0.08*** -0.07** -0.08*** -0.08***  -0.11*** -0.15*** 
(0.04) (0.03) (0.03) (0.02) (0.02) (0.04) (0.03) (0.04) (0.01) (0.02) 
Take credit-bearing math by end of 1st year 0.00 -0.01 -0.02 0.01 0.04** 0.04 0.03 0.04 0.04** 0.07*** 
(0.04) (0.03) (0.03) (0.02) (0.02) (0.04) (0.03) (0.03) (0.02) (0.02) 
Pass credit-bearing math by end of 1st year -0.03 -0.04 -0.05 -0.03 -0.01 -0.02 -0.01 -0.02 -0.02 -0.02 
(0.04) (0.03) (0.03) (0.02) (0.02) (0.03) (0.03) (0.03) (0.02) (0.02) 
Free/reduced price lunch eligible students 
Take remedial math by end of 1st year -0.02 -0.10** -0.11*** = -0.13*** -0.11*** -0.09** -0.11*** -0.11*** = -0.14*** -0.15*** 
(0.05) (0.04) (0.04) (0.02) (0.02) (0.04) (0.04) (0.03) (0.02) (0.03) 
Take credit-bearing math by end of 1st year -0.04 -0.02 -0.00 0.07** 0.10*** 0.09** 0.12*** 0.12*** 0.10*** 0.08*** 
(0.05) (0.04) (0.04) (0.03) (0.02) (0.05) (0.04) (0.04) (0.02) (0.03) 
Pass credit-bearing math by end of 1st year -0.08* -0.08* -0.07* -0.05* -0.03 0.09** 0.09***  0.09*** 0.05** 0.03 
(0.05) (0.04) (0.04) (0.03) (0.03) (0.04) (0.04) (0.03) (0.02) (0.02) 
Black or Hispanic students 
Take remedial math by end of 1st year -0.04 -0.10 -0.07 -0.11** -0.11** 0.03 -0.02 -0.03 -0.10*** -0.10*** 
(0.10) (0.08) (0.07) (0.05) (0.04) (0.06) (0.05) (0.05) (0.03) (0.03) 
Take credit-bearing math by end of 1st year 0.12 0.12 0.11 0.11** 0.13*** 0.02 0.02 0.02 0.04 0.08** 
(0.10) (0.08) (0.07) (0.05) (0.04) (0.06) (0.06) (0.05) (0.03) (0.04) 
Pass credit-bearing math by end of 1st year 0.00 0.01 0.03 0.03 0.04 -0.02 0.01 0.00 0.01 0.01 
(0.10) (0.09) (0.08) (0.06) (0.05) (0.06) (0.05) (0.05) (0.03) (0.03) 
Top quartile school in % students below cutoff 
Take remedial math by end of 1st year -0.07 -0.16*** -0.17*** = -0.15*** -0.08*** -0.09* -0.07* -0.06 -0.12*** -0.21*** 
(0.06) (0.05) (0.04) (0.03) (0.03) (0.05) (0.04) (0.04) (0.02) (0.03) 
Take credit-bearing math by end of 1st year -0.01 -0.01 0.01 0.05 0.09*** -0.05 -0.00 -0.00 0.05* 0.07** 
(0.06) (0.05) (0.05) (0.03) (0.03) (0.06) (0.05) (0.05) (0.03) (0.03) 
Pass credit-bearing math by end of 1st year -0.07 -0.08 -0.08* -0.05 -0.02 -0.04 -0.02 -0.04 0.01 0.04 
(0.06) (0.05) (0.05) (0.03) (0.03) (0.05) (0.05) (0.04) (0.03) (0.03) 


Note: *, ** and *** denote statistical significance at the 0.10, 0.05 and 0.01 levels, respectively. Standard errors in parentheses. Bold and italic estimates are 
significantly different from the corresponding effects among all students. Difference-in-RD estimates assume linear association between the running variable and 
outcomes. DD estimates use students scoring up to three points above the cutoff as the comparison and control for student ACT scores, FRL-eligibility, gender and 


race/ethnicity. 
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Table 11. Estimated Effect of Targeted Interventions (TI) in English on College Outcomes, by Student and School Characteristics 


2 year 4 year 
Difference-in-RD DD Difference-in-RD 
BW=3 BW=4 BW=5 ACT 15-17. =ACT 12-14 BW=3 BW=4 BW=5 ACT 15-17 ACT 12-14 
Students who missed ACT cutoff in addition to English 
Take remedial English by end of 1st year -0.02 -0.04* -0.04** -0.10*** -0.14*** 0.00 -0.03* -0.03** -0.10*** -0.15*** 
(0.02) (0.02) (0.02) (0.01) (0.01) (0.02) (0.02) (0.02) (0.01) (0.01) 
Take credit-bearing English by end of 1st year 0.04 -0.01 -0.00 0.02* O:077** -0.00 -0.00 0.01 0.03*** 0.06*** 
(0.03) (0.02) (0.02) (0.01) (0.01) (0.02) (0.02) (0.02) (0.01) (0.01) 
Pass credit-bearing English by end of 1st year 0.04 0.01 -0.00 0.01 0.05*** -0.03 -0.04 -0.01 -0.01 0.03 
(0.03) (0.03) (0.02) (0.01) (0.01) (0.03) (0.03) (0.02) (0.01) (0.02) 
Free/reduced price lunch eligible students 
Take remedial English by end of 1st year -0.01 -0.01 -0.02 -0.10*** -0.14*** -0.04 -0.11*** -0.09*** -0.14*** -0.16*** 
(0.04) (0.03) (0.03) (0.01) (0.02) (0.03) (0.03) (0.02) (0.01) (0.02) 
Take credit-bearing English by end of 1st year 0.07* 0.04 0.02 0.04** 0.08*** 0.03 0.02 0.04 0.03* 0.05** 
(0.04) (0.03) (0.03) (0.02) (0.02) (0.04) (0.03) (0.03) (0.02) (0.02) 
Pass credit-bearing English by end of 1st year 0.04 0.02 0.01 0.01 0.03 -0.00 -0.03 0.01 -0.02 0.00 
(0.05) (0.04) (0.03) (0.02) (0.02) (0.05) (0.04) (0.04) (0.02) (0.02) 
Black or Hispanic students 
Take remedial English by end of 1st year -0.18** = -0.19*** = -0.14*** = -0,.14*** -0.11*** 0.05 0.00 0.01 -0.07*** -0.11*** 
(0.07) (0.06) (0.05) (0.03) (0.03) (0.04) (0.03) (0.03) (0.02) (0.02) 
Take credit-bearing English by end of 1st year 0.24*** 0.13** 0.12** 0.09*** 0.12*** -0.01 -0.05 -0.02 0.00 0.07*** 
(0.08) (0.06) (0.06) (0.03) (0.03) (0.05) (0.04) (0.04) (0.02) (0.03) 
Pass credit-bearing English by end of 1st year 0.13 0.08 0.06 0.04 0.09** 0.06 0.02 0.07 0.01 0.04 
(0.09) (0.07) (0.07) (0.04) (0.04) (0.07) (0.06) (0.05) (0.03) (0.03) 
Top quartile school in % students below cutoff 
Take remedial English by end of 1st year -0.05 -0.07* -0.05 -0.12*** -0.15*** -0.00 -0.04 -0.06** -0.12*** -0.14*** 
(0.05) (0.04) (0.03) (0.02) (0.02) (0.04) (0.03) (0.03) (0.02) (0.02) 
Take credit-bearing English by end of 1st year 0.02 0.01 0.02 0.03 0.06*** 0.07 0.06 0.07** 0.05** 0.07*** 
(0.05) (0.04) (0.04) (0.02) (0.02) (0.04) (0.04) (0.03) (0.02) (0.02) 
Pass credit-bearing English by end of 1st year -0.02 -0.03 -0.01 0.01 0.05* -0.04 -0.02 -0.00 -0.02 -0.00 
(0.06) (0.05) (0.04) (0.03) (0.03) (0.06) (0.05) (0.04) (0.03) (0.03) 


Note: *, ** and *** denote statistical significance at the 0.10, 0.05 and 0.01 levels, respectively. Standard errors in parentheses. Bold and italic estimates are significantly 
different from the corresponding effects among all students. Difference-in-RD estimates assume linear association between the running variable and outcomes. DD 


estimates use students scoring up to three points above the cutoff as the comparison and control for student ACT scores, FRL-eligibility, gender and race/ethnicity. 
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Table 12. Correlation Between College Outcomes and Intervention Type, by Institution 
Type and Intervention Subject: 2014 and 2015 Cohorts of 11th Grade Students 


2 year 4 year 
Dependent Variable ESS Other ESS Other 
Math intervention in high school 
Take remedial math by end of 1st year 0.02 0.00 -0.02 -0.02 
(0.02) (0.03) (0.02) (0.02) 
Take credit-bearing math by end of 1st year 0.01 0.01 0.03 0.02 
(0.02) (0.02) (0.02) (0.02) 
Pass credit-bearing math by end of 1st year 0.00 0.02 0.04* 0.02 
(0.02) (0.02) (0.02) (0.02) 
English intervention in high school 
Take remedial English by end of 1st year 0.05* 0.02 0.03* -0.02* 
(0.02) (0.02) (0.02) (0.02) 
Take credit-bearing English by end of 1st year -0.04 0.01 0.00 -0.00 
(0.02) (0.02) (0.02) (0.02) 
Pass credit-bearing English by end of 1st year -0.01 0.04* 0.01 -0.02 
(0.02) (0.02) (0.03) (0.03) 


Note: Estimated coefficients on intervention type (ESS and other) reported in table with standard errors in parentheses. 
Standard errors are clustered at the school level. The regression samples include all TI participants from the 2014 and 
2015 cohorts of 11th grade students for whom intervention data are available. College outcomes are regressed on 
indicator variables for intervention type (ESS or other, with transition courses as the reference intervention type) and 
control variables that include student ACT scores, FRL eligibility, gender and race/ethnicity. ** and *** denote statistical 
significance at the 0.05 and 0.01 levels, respectively. 
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